Search for: All records

Preferences within a group of people are not uniform but follow a distribution. While existing alignment methods like Direct Preference Optimization (DPO) attempt to steer models to reflect human preferences, they struggle to capture the distributional pluralistic preferences within a group. These methods often skew toward dominant preferences, overlooking the diversity of opinions, especially when conflicting preferences arise. To address this issue, we propose Group Distributional Preference Optimization (GDPO), a novel framework that aligns language models with the distribution of preferences within a group by incorporating the concept of beliefs that shape individual preferences. GDPO calibrates a language model using statistical estimation of the group's belief distribution and aligns the model with belief-conditioned preferences, offering a more inclusive alignment framework than traditional methods. In experiments using both synthetic controllable opinion generation and real-world movie review datasets, we show that DPO fails to align with the targeted belief distributions, while GDPO consistently reduces this alignment gap during training. Additionally, our evaluation metrics demonstrate that GDPO outperforms existing approaches in aligning with group distributional preferences, marking a significant advance in pluralistic alignment. 

Abstract The export of the North Atlantic Deep Water (NADW) from the subpolar North Atlantic is known to affect the variability in the lower limb of the Atlantic meridional overturning circulation (AMOC). However, the respective impact from the transport in the upper NADW (UNADW) and lower NADW (LNADW) layers, and from the various transport branches through the boundary and interior flows, on the subpolar overturning variability remains elusive. To address this, the spatiotemporal characteristics of the circulation of NADW throughout the eastern subpolar basins are examined, mainly based on the 2014–20 observations from the transatlantic Overturning in the Subpolar North Atlantic Program (OSNAP) array. It reveals that the time-mean transport within the overturning’s lower limb across the eastern subpolar gyre [−13.0 ± 0.5 Sv (1 Sv ≡ 10⁶m³s⁻¹)] mostly occurs in the LNADW layer (−9.4 Sv or 72% of the mean), while the lower limb variability is mainly concentrated in the UNADW layer (57% of the total variance). This analysis further demonstrates a dominant role in the lower limb variability by coherent intraseasonal changes across the region that result from a basinwide barotropic response to changing wind fields. By comparison, there is just a weak seasonal cycle in the flows along the western boundary of the basins, in response to the surface buoyancy-induced water mass transformation.